For some time various companies have been actively developing auto-stereoscopic displays suitable for rendering three-dimensional (3D) imagery. Autostereoscopic devices can present viewers with a 3D impression without the need for special headgear and/or glasses.
Autostereoscopic displays generally provide different views for different viewing angles. In this manner a first image can be generated for the left eye and a second image for the right eye of a viewer. By displaying appropriate images, i.e. appropriate from the viewpoint of the left and right eye respectively, it is possible to convey a 3D impression to the viewer.
A variety of techniques are used to generate images for autostereoscopic displays. For example, multi-view images can be recorded using multiple cameras, wherein the position of the respective camera corresponds with the respective viewpoint of each respective view.
In order to maintain backwards compatibility and improve on bandwidth usage many of the current autostereoscopic displays use an input signal comprising a sequence of conventional two-dimensional (2D) images and corresponding depth-maps.
Depth-maps provide depth information indicative of the absolute or relative distance of objects depicted in the image to the camera. By way of example, 8-bit grey-scale images are commonly used to represent depth information. Depth-maps can provide depth information on a per-pixel basis, but as will be clear to the skilled person may also use a coarser granularity, such as a lower resolution depth-map wherein each depth-map value provides depth information for multiple pixels.
Disparity maps can be used as an alternative to the above mentioned depth-maps. Disparity refers to the apparent shift of objects in a scene when observed from two different viewpoints, such as from the left-eye and the right-eye viewpoint. Disparity information and depth information are related and can be mapped onto one another provided the geometry of the respective viewpoints of the disparity map are known, as is commonly known to those skilled in the art.
In view of this close relationship and the fact that one can be transformed into the other, the term “depth-map” and “depth information” used throughout the description are understood to comprise depth information as well as disparity information.
By providing an autostereoscopic display with an image sequence and a corresponding sequence of depth information maps, or depth-maps for short, the autostereoscopic display can render multiple views of the content for one or more viewers. In the above manner a conventional signal is enhanced with a depth-map.
In order to improve the quality of multi-view rendering using a 2D+depth signal, additional occlusion information also referred to as de-occlusion information, may be provided. (De-)occlusion information relates to image and/or depth information which can be used to render views for viewpoints other than those of the 2D+depth information provided. When rendering a view based on the 2D+depth information for a viewpoint that differs from that of the 2D+depth information, information may be required that is not present in the original 2D+depth information. This information may be provided in the occlusion information; in addition to the information that was occluded by objects, the occlusion information may also comprise information in the vicinity of occluded regions. The availability of occlusion information enables filling in of holes which occur when rendering views using a 2D+depth signal. Throughout the application the term occlusion information is understood to comprise occluded image information and/or occluded depth information that can be used for filling in de-occluded regions in the view-rendering process.
International Application WO2006/137000 discloses a method of combined exchange of image data and further data, such as occlusion data. Although the above format is particularly well suited for the exchange of 3D content between e.g. a set-top box (STB) and a display, there is a need to further reduce the size of such 3D content when transmitting or distributing such content over a bandwidth-limited medium. Although lossy compression algorithms, such as lossy DCT-based compression algorithms, may provide a significant reduction in size, they also tend to introduce noise. “Compression artifacts in 3D television signals” by Chris Varekamp, presented at the second annual IEEE BENELUX/DSP Valley Signal Processing Symposium (SPS-DARTS 2006) shows that the effect of noise in a depth-map can, as a result of the multi-view rendering process, result in serious parallax errors, in particular on object boundaries. This effect is a direct result of the fact that the compression algorithm is tuned to reduce the perceived error in the depth-map rather than in the resulting rendered multi-view images.